Summary

This PR adds PyTorch CUDA/Triton FP8 KV-cache support with a concise public policy surface aligned with vLLM-style normal FP8 behavior.

Supported policies:

• fp8 / fp8_e4m3 -> scalar-scale torch.float8_e4m3fn KV cache
• fp8_e5m2 -> scalar-scale torch.float8_e5m2 KV cache

The implementation intentionally does not expose per-attention-head / per-token-head FP8 KV-cache modes or dynamic calculate_kv_scales-style calibration. Those paths were removed from the public surface to keep this PR focused on the normal scalar-scale FP8 path.

What Changed

Public API / Config

• Adds QuantPolicy.FP8 for E4M3 scalar-scale FP8 KV cache.
• Adds QuantPolicy.FP8_E5M2 for E5M2 scalar-scale FP8 KV cache.
• Updates CLI parsing so:
  • --quant-policy fp8 defaults to E4M3
  • --quant-policy fp8_e4m3 maps to E4M3
  • --quant-policy fp8_e5m2 maps to E5M2
• Keeps unsupported/deprecated aliases out of the public CLI surface.
• Keeps deprecated dynamic scale calculation and per-attention-head/per-token-head policies out of this PR.

Cache Allocation

• Stores normal FP8 KV cache payloads directly as torch.float8_e4m3fn or torch.float8_e5m2.
• Avoids allocating per-token/head FP8 scale metadata for normal FP8.
• Keeps existing INT4/INT8/TurboQuant metadata behavior unchanged.

Attention Runtime

• Adds scalar k_scale / v_scale buffers on PyTorch attention layers.
• Defaults scalar scales to 1.0.
• Warns once when using normal E4M3 FP8 with default k_scale=v_scale=1.0.
• Threads scalar scales through the PyTorch CUDA attention path for cache fill, flatten, prefill, and decode.

Kernels

• Adds scalar-scale FP8 cache fill:
  • quantizes as value / scale
  • clamps to the target FP8 dtype range
  • stores directly into FP8 cache tensors
• Adds scalar-scale FP8 flatten:
  • reads FP8 cache
  • dequantizes as stored * scale
  • writes contiguous K/V states for prefill paths
• Adds scalar-scale FP8 paged decode support:
  • applies k_scale in QK score computation
  • applies v_scale before the PV accumulation
  • avoids materializing full dequantized K/V tensors in decode

Tests

• Adds CLI/config tests for FP8 policy parsing and dtype mapping.
• Adds cache descriptor tests confirming normal FP8 does not allocate extra quant metadata.
• Adds kernel tests for:
  • FP8 scalar cache fill
  • FP8 scalar cache flatten
  • FP8 scalar paged attention decode
  • E4M3 and E5M2 variants
• Adds SM80/SM90-aware guards for Triton FP8 dtype support differences.
• Extends existing quant-policy pipeline tests for FP8 behavior while avoiding known unsupported architecture paths.

Scope

This PR focuses on the LMDeploy PyTorch CUDA/Triton attention path.

Out of scope:

• TurboMind FP8 KV-cache support
• dlinfer/Ascend FP8 KV-cache support
• per-attention-head or per-token-head FP8 KV-cache quantization
• dynamic runtime KV-scale calibration
• dataset/llm-compressor calibration workflows
• FlashMLA-specific FP8 behavior beyond existing guarded paths

Rationale

The previous experimental direction included per-token/head scale metadata and dynamic scale calculation. That made the implementation heavier and less aligned with the common normal FP8 KV-cache path used by vLLM.

This PR keeps the first upstreamable FP8 KV-cache feature smaller and easier to validate:

• normal FP8 uses scalar layer-level scales,
• no per-token/head scale metadata is allocated,
• no per-step absmax scale calculation runs on the hot path,
• cache fill/decode/flatten behavior is explicit and test-covered,
• unsupported or deprecated modes are not exposed as public API.

Test Plan

python -m pytest -q tests/test_lmdeploy/test_fp8_kv_cache_policy.py
python -m pytest -q tests/pytorch/kernel/test_fill_kv_cache.py
python -m pytest -q tests/pytorch/kernel/test_flatten_kv_cache.py
python -m pytest -q tests/pytorch/kernel/test_paged_attention.py
python -m pytest -q tests/test_lmdeploy/test_quant_policy.py